{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "逻辑回归的原理：  \n",
    "https://windmising.gitbook.io/liu-yu-bo-play-with-machine-learning/9-1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logRegres\n",
    "import utils"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 批量梯度下降法\n",
    "\n",
    "目标函数的偏导公式为：\n",
    "\n",
    "$$\n",
    "\\nabla J(\\theta) = \\frac{1}{m} \\cdot \\\n",
    "\\begin{Bmatrix}\n",
    "\\sum_{i=1}^m (\\hat y^{(i)}-y^{(i)}) \\\\\n",
    "\\sum_{i=1}^m (\\hat y^{(i)}-y^{(i)})\\cdot X_1^{(i)} \\\\\n",
    "\\sum_{i=1}^m (\\hat y^{(i)}-y^{(i)})\\cdot X_2^{(i)} \\\\\n",
    "... \\\\\n",
    "\\sum_{i=1}^m (\\hat y^{(i)}-y^{(i)})\\cdot X_n^{(i)} \\\\\n",
    "\\end{Bmatrix}   \\\n",
    "= \\frac{1}{m}\\cdot X_b^T\\cdot (\\sigma(X_b\\theta)-y)\n",
    "$$\n",
    "\n",
    "公式推导详见：https://windmising.gitbook.io/liu-yu-bo-play-with-machine-learning/9-1/9-3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "coeff =  [[ 4.12414349]\n",
      " [ 0.48007329]\n",
      " [-0.6168482 ]]\n"
     ]
    }
   ],
   "source": [
    "train_X,train_y=utils.loadDataSet()\n",
    "coeff = logRegres.gradAscent(train_X,train_y)\n",
    "print (\"coeff = \", coeff)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expect Result:  \n",
    "\n",
    "coeff =  [[ 4.12414349]  \n",
    " [ 0.48007329]  \n",
    " [-0.6168482 ]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de5SddX3v8fc3NxiSTC4wCTO5IsSQAQLINOqhrQICAbFQjwehZymn2uboqkfltCxQe9RD2yWWWi+livRA1R6rUIVIlavQs1CWCMMdctFwCeQCCeROEnKZ7/lj75ns2bP3M/vyXH7P3p/XWlmZ/exnZr5ze7779/3+fr/H3B0REZFqxmQdgIiIhE2JQkREIilRiIhIJCUKERGJpEQhIiKRxmUdQBKOOuoonz9/ftZhiIjkxqOPPvqau3dVeq4lE8X8+fPp7+/POgwRkdwws7XVnlPpSUREIilRiIhIpMQThZndZGabzOyZkmNfNLP1ZvZE8d/5Vd53qZmtNrM1ZnZV0rGKiMhIaYwovgMsrXD8q+5+SvHfHeVPmtlY4B+B84Be4FIz6000UhERGSHxROHuDwBbGnjXJcAad3/e3fcBPwQujDU4EREZVZaznj5hZh8G+oE/d/etZc/PAl4uebwOeHu1D2Zmy4BlAHPnzo05VJH8Wf74eq69ezUbtu2hZ2oHV5y7kItOnZV1WJJDWTWzvwUcC5wCbAS+UuEcq3Cs6la37n6Du/e5e19XV8WpwCJtY/nj6/nMrU+zftseHFi/bQ+fufVplj++PuvQJIcySRTu/qq7H3T3AeCfKJSZyq0D5pQ8ng1sSCM+kby79u7V7Nl/cNixPfsPcu3dqzOKSPIsk0RhZt0lD/8QeKbCaY8AC8zsGDObAFwC3J5GfCJ5t2HbnrqOi0RJY3rsD4BfAQvNbJ2ZfRT4WzN72syeAs4ALi+e22NmdwC4+wHgE8DdwErgFnd/Nul4RVpBz9SOuo6LREm8me3ul1Y4fGOVczcA55c8vgMYMXVWRKJdce5CPnPr08PKTx3jx3LFuQszjEryqiX3ehJpd4OzmzTrSeKgRCHSoi46dZYSg8RCiUIkYFoLISFQohAJ1OBaiME+w+BaCEDJQlKl3WNFAqW1EBIKjShEAlJaaqq2DUEIayFUEmsvShQigSgvNVWT9VoIlcTaj0pPIoGoVGoqF8JaCJXE2o9GFCKBiCopGQRT4tH2IO1HiUIkED1TO1hf4WI7a2oHD151ZgYRVVYtzqxLYpIclZ5EAnHFuQvpGD922LEQSk3l8hKnxEcjCpFA5GXbjTTi1KyqsJh71XsB5VZfX5/39/dnHYaINKDS7K+O8WP50vtPUrJIkJk96u59lZ5T6UlEgqJZVeFRohCRoGhWVXiUKEQkKLrpUniUKESkouWPr+f0a+7nmKt+xunX3M/yx9en8nk1qyo8mvUkIiNkuU1HkrOqNJuqMRpRiMgIWTeULzp1Fg9edSZf/eApAFx+8xNNj2oGk9/64oaLg8kvrZFSniWeKMzsJjPbZGbPlBy71sxWmdlTZnabmU2t8r4vmtnTZvaEmWm+q0hKQmgox31hzzr55VkaI4rvAEvLjt0LnOjui4HfAJ+JeP8z3P2UavN7RSRaI72GEBrKcV/YQ0h+eZV4onD3B4AtZcfucfcDxYcPAbOTjkOkHf3l8qe5/OYn6n5VHkJDOe4LewjJL69C6FF8BLizynMO3GNmj5rZsqgPYmbLzKzfzPo3b94ce5AiebP88fV8/6GXRtwAqZZX5RedOosvvf8kZk3twChsTJj2yui4L+whJL+8ynTWk5l9DjgAfL/KKae7+wYzmwHca2ariiOUEdz9BuAGKGzhkUjAkijNSInXtXevbuoueRedOivT7/8V5y6suJVHoxf2vOylFaLMEoWZXQZcAJzlVTaccvcNxf83mdltwBKgYqKQfNNd0+IXlQymdIzn9GvuD/qCmcSFPevkl1eZJAozWwpcCbzL3XdXOWciMMbddxbfPge4OsUwJUVRjUv9YTem2n0jAN7Yd4Bte/YDYSdlXdjDkMb02B8AvwIWmtk6M/socB0wmUI56Qkzu754bo+Z3VF815nAL83sSeBh4GfuflfS8Uo2NCMlfpVq8gZMnDCW/QeHD+I1TVSiJD6icPdLKxy+scq5G4Dzi28/D5ycYGgSkLzdNS0P/ZRqpZvLb36i4vlKylKNtvCQIMTduExSnvoplUo31969OldJWbKnRCFBSHNGyvLH1/PF258dqtFPO2I8X3jfCTV/rrz3U0JKyuUjszOO7+I/Vm0OeqTWjpQoJBhpNC6XP76eK/7tSfYPHKrRb929nyt+9ORQDKPJez8llGmilUZm//ehl4aeD3mk1m6UKKStXHv36mFJYtD+g17ziCBv/ZRKQphNVGlkVi5PI7VWFsLKbJGm1bqfUdSr/lpHBFrhG49av995Gam1MiUKyb16dhmNetVf64gghO0tWkGt3+88jdRalUpPknv1NJevOHfhiB4FwPixVteIIITSTd5VaqqX00gtDEoUknv1NJcHL+5Rs55CXiPRbGwhfW2Vmuqa9RQmq7LNUq719fV5f7/uc9QuTr/m/orN5VlTO3jwqjPr+ljlM3Gg8Ko2hNJSs7FVen+jsEXzLF2U256ZPVrtvj/qUUjuxdlcDvkuaM3GVun9B18m6ragEkWJQnIvzuZyyGskmo1ttPNCSYgSHvUopCXE1VxOYo1EXH2BZmOL2k12UFQyCam/IenSiEJyrZH7QUeJe41EPVN3k46t0vuXq5Z04vw6JH+UKCS3krh4xb1GIs6eR7Oxlb4/FBrZpaKSTsi9G0meSk+SW0ltzhfnGolm+grVSj1xfW31lJJC7t1I8pQoJLeyuHjVW6efesR4tu7eP+L4aH2FNLYyryfptML+VqA+S6NUepLcqnaRSuriVW+pa/nj69m198CI47WsAg+t1NMK+1upz9I4JQrJrbQvXvVevKvtVDtxwrhRX8WGVupphf2tQku+eZJK6cnMbgIuADa5+4nFY9OBm4H5wIvAxe6+tcL7Xgb8ZfHhX7v7d9OIWcKX9n0V6r14Vzu+fc/IUlS5EEs9ed/fKrTkmydpjSi+AywtO3YVcJ+7LwDuKz4epphMvgC8HVgCfMHMpiUbquTJRafO4sGrzuSFa97Lg1edmeiFrN5SVzOlsVYo9YQm7VJlK0klUbj7A8CWssMXAoOjg+8CF1V413OBe919S3G0cS8jE45IKuq9eDdzsU+r1BP3OpSQKfk2LstZTzPdfSOAu280sxkVzpkFvFzyeF3xmEjq6i11NVsaq7XU0+hMnjRmVoUklFvA5lFqu8ea2XzgpyU9im3uPrXk+a3uPq3sfa4ADnP3vy4+/l/Abnf/SoWPvwxYBjB37tzT1q5dm9SXIhKMZnaUjXPXXcm/UHePfdXMugGK/2+qcM46YE7J49nAhkofzN1vcPc+d+/r6uqKPViREDUzk6daE3f9tj2plaBqKX21U3ksVFkmituBy4pvXwb8pMI5dwPnmNm0YhP7nOIxCYz+mLMRdbEf7WcR1cRNY31BLesatPYhDKkkCjP7AfArYKGZrTOzjwLXAGeb2W+Bs4uPMbM+M/s/AO6+Bfgr4JHiv6uLxyQg+mPOTtTFfrSfRdQmgWmsL6hlNKS1D2FIa9bTpe7e7e7j3X22u9/o7q+7+1nuvqD4/5biuf3u/icl73uTux9X/PfPacQr9dEfc3Zq2RG20s9isAEedb/qpNcX1LKuodq26Fr7kC7t9SRN00Km7JTP5Kk2NaX0Z1GpAV5J0usLRltUuPzx9UO3ak07NhlOiUKaVu0PfuoR4zn9mvtzOxUxLxvIlU6jrTaTqfTCOtpIAtJZX3DFuQsrztga/LzX3r26YpKw4vtKerTXkzStUvlj/Fhj194Due1bhNJ3qXeSQC2LyqJGemnu4zTaosJqcTqtuc4jZBpRSNMqLWR6480DbCvb0yiOe0WkJal7XdSjkQVxtSwqqzYCzGL9RNSiwqg4JV1KFBKL8j/4Y676WcXz8tK3CKHv0miyGm1F92gln1DkJc6s7d53gFWv7GTFhh28be40ens6Y/8cShSSiBB3P61HCPEnlazyspVFXuJM06ade1mxYQcrNu4Y+v+F195gcIONK85dqEQh+ZH3V4MhxF9rsmqk6Z6XLcPzEmfcDg44L7z2xrCEsGLDDl7b9ebQOXOmd9Db3cmFJ8+it6eT3p5OeqYcnkg8ShSSiLy/Ggwh/lqSVbtt7NeKSktHgwlh1Ss72Lt/AChMDHnrzMmcsbCrkBC6Ozm+u5MpHeNTizG1TQHT1NfX5/39/VmHIdK00UYL2tgvX0YrHU3pGE9vd+dQQujt6eTYrklMGJf8BNWoTQE1ohAJ2GillxCa7jJSM6UjM6v/Ez51C9x3NWxfB1Nmw1mfh8UXx/b1KFGI5FhsTfeELzStbM++g6x8ZUdk6WjBjARLR0/dAv/+Sdhf/D3Y/nLhMcT2M1SiEMmxWJruKVxoWsXmnW+WjRK288JrbzBQLB11Hj6O3p5O/mjJvKGkcNyMhEtH91196Gc3aP+ewnElChGJpemewoUmbw4OOC++/saIfsLmnYdKR7OnFUpH7zu5Z6ifMGtqR2Olo2ZsX1ff8QYoUYgkIM19opqeQprChSZke/YdZNUrwxPCqo07h0Zpg6Wjd721ayghLEp51lGkKbMLo8BKx2OiRCESs9xNWU3hQhOKWktHly6Zm17pqFlnfX546RBgfEfheEyUKERiFsI+UXVJ4UKTtlyVjpo1WB7UrCeR/MjdlNUULjRJaqh0dHQnU44IpHQUh8UXJ/rzUqIQiVkI+0TVLeELTVw273yTlRuHJ4XnN+8aKh1NPnwcvd2dXLJkzlBSWDBjMhNW/Aju+zg8m79EGAIlCpGYhbBPVAiaaegfHHDWvj5ywdqmktLRrKkd9PZ08t6Tuof6CbOnVSgdafpv0zJLFGa2ELi55NBbgM+7+9dKznk38BPgheKhW9396tSCFGlACPtEZa2ehv6efQdZ/ergXkfbiwvWdrJ7X+F9x40xFsyczO8tOLRgrbe7jtKRpv82LbNE4e6rgVMAzGwssB64rcKpv3D3C9KMTaRZ7brr6aBqDf1r7lzF9IkTRi0dffB3DpWOjpsxicPGja3wWWrU5tN/4xBK6eks4Dl3X5t1ICLSvEo9GoBXduzlwzc9DNRROmpWG03/TUooieIS4AdVnnunmT0JbAD+wt2frXSSmS0DlgHMnTs3kSBFZKRKpSOjcG/rclMOH8/1HzqNRd2TmXrEhHQCbMHpv2nLfJtxM5tAIQmc4O6vlj3XCQy4+y4zOx/4ursvGO1japvx1pXmimcZ6bVdb45YmzCsdHTYOBb1dHL4uDH86vnX2X/w0PWlY/xYvvT+k7L5eWnTw1GFvs34ecBj5UkCwN13lLx9h5l908yOcvfXUo1QgpC7Fc+hqeNiOTDgrN2ye9goYcXGHby6Y/iso0XdnZx/Uje93Z2c0DO8dBRUUs/J9N9QhZAoLqVK2cnMjgZedXc3syXAGOD1NIOTcORuxXNIIqaI7l30n1n9ys5ho4SVG3cMm3V03IxJnH7cUcNuqjN1zXK475PwQknimX7oYtzuDf1WkmmiMLMjgLOB/15y7GMA7n498AHg42Z2ANgDXOJZ18rypoWG3Llb8ZyURn6mxSmir/tkVgzMY4XPZ8W+eay4eRfPDdw1onR0cd+coYSwYGaFWUdam9BWMk0U7r4bOLLs2PUlb18HXJd2XC2jxf6Yc7niOW41/kxHlI42X8KKgXm8yvShc2axmUVjXuK8M3+3YukoktYmtJUQSk+SlBb7Y9aKZyr+TPfuO8Dqu25ixd53VCwdjR1jLBhzNKfzLL1j1tJrL7JozEtMs10wZQ6c3cDsH61NaCtKFK2sxf6YteIZXt+2nRUDJxZKRwPzWOHzeM57GHhzDNz6NJMOKyxYu7hv+IK1w1f+GP79O/FNEdXahLaiRNHKWvCPOa0GadYzdqrOOnpzqDJLD6/RO2Yt5415mN5Ju+hd9s/MntbBmDEVSkdx7xCrtQmNyWnPUImilemPuSFpT8Pdu/8gvxlasFaYebRy4w7eKC0dzZjE6cceRS/P07vqGywa+G2hdASFn+l7vwFHHhH9ieKcIprzrckzkeOeYeYL7pKgBXclcvoKJkunX3N/xab5rKkdPHjVmU197Nd3vcnKjTuHjRKe2/wGB4vTjgZLR0Ob3w2WjsaXzDrSzzSfvnpilRH+HLj8mfTjKRP6gjtJkhYa1S2OabgDA85LW3aP2Cb7lR17h87pnnI4vd2dnHvC0ZzQ00lv95TqpaNS+pnmU457hkoUImXqnYZbS+nouK5JvPPYIw/dYa27k+kTR9nrqBVGDq3wNcQlxz1DJQoJRtYN5EFR03C3vLFvRIO5vHS0qHsyHzhtdrF8NIUFM8tKR7XIcT17SCt8DXHKcc9QPQoJQnkDGbLdRO7Wx9bx5TtX8erON5l82DjmTD+CLW/sq1g6Ku0nzJl2xOilo1o0Us8O7dV74DX5TIT2MyqhHoUEL8t9nPbuP8hvX901NEp4tqx0tHv/QQ4OeP2lo2bUW88O8dV7jmvyiclpfykyURS3+e5y9+fKji9296cSjUzaSlr7OG15Yx8ryxrMazbvGiodTZwwlt6eztpKR0m+Oqy3nh3iKvyOabBny8jjOajJy3BVE4WZXQx8DdhkZuOB/+bujxSf/g7wtuTDk3YR9z5OAwPOy1t3j7h3wsbth0pHR3ceTm9PJ2f3zhwqH82dXmPpKOlX8PXWs0N79f7ULfDmzpHHx07IRU1ehosaUXwWOM3dNxa3+P4XM/usu98KxHyvQml3zezjVF46Kux1tJNdbx4ACrOOju2ayNuPmT40SljUPZkjJx3WeMBJv4Kvd0FbaDNq7rsaBvaPPD5hUi5LL+0uKlGMc/eNAO7+sJmdAfzUzGZT+S6HIg2rdR+nrW/sG7E2obx0tKi7k/e/bdZQP+GtMyfXP+toNGm8gq+nnh3ajJpq34c9W9ONQ2IRlSh2mNmxg/2J4sji3cBy4IQ0gpP2UrqP02Dp6M6nNyZTOmpWaK/gQ9tSI6TvT8AzjfIiKlFcSVmJyd13mtlS4DOJRiVtZe/+g6zZtKs442j7iNLRGIPjZkyKt3TUrNBewUNYM2pC+f6EOBssh6ISxXeBb5vZV9z9AICZzQS+AiwErk4hPmkxWwdnHZWMEtZs2sWBtEtHzQrtFXxoQvn+hDgbLIeiEsVpwJeAx83sU8BJwP8E/hb4cAqxSY65Oy9v2TOswbxiww42lJSOZnYeRm93J2ctmkFv9xRO6EmxdBSHkF7BhyiE709os8FyqmqicPetwMeKSeLnwAbgHe6u77AM8+aB4qyjsr2OdpaUjo7tmsTvHDN92IK1o7IsHUl+1dNzaKZXot7GkKh1FFOBLwNvB5YC5wN3mtmn3P3+uAIwsxeBncBB4ED5EnIr3MD368XPv5vCeo7H4vr8Up/RSkdHFEtHF506a6jBvPDoAEtHkr1GLsT19hwa7ZWotzFMVOnpMeCbwJ8VexT3mNkpwDfNbK27XxpjHGe4+2tVnjsPWFD893bgW8X/JUGNlI56ezqZl6fSkWSn0QtxvT2HRnsl6m0ME5Uofr+8zOTuTwD/ycz+NNmwhrkQ+J4Xdi98yMymmln34BoPaZ5KRwlS+aKyRi/EjfQcGumVqLcxTFSPoup3xN3/KcYYnMJoxYFvu/sNZc/PAkqLjOuKx4YlCjNbBiwDmDt3bozhtZZtuyssWCsrHR1/9GQuPLVnaJSwcOZkOiZkWDrK68VW5YvqGr0Qp7U+I6R1IAEIYffY0919g5nNAO41s1Xu/kDJ85XqGCNWhhcTzA1Q2GY8mVDzw91Zt3UPz5aNEkr3U5ox+TB6ezo58/gZQ/2EeUdOZGxIpaM8X2xVvqiu0QtxWuszQlkHEojME4W7byj+v8nMbgOWAKWJYh0wp+TxbAozsKRoqHRUMlJYuWF46egtXZM4bd40PvTOefR2F0pHXZNzUDrK88VW5YvqGr0Qp7U+I5R1IIHINFGY2URgTHHF90TgHEYu5Lsd+ISZ/ZBCE3t7O/cnRisddYwfy6LuwEpHzcjzxVbli+qauRCntT4jhHUggch6RDETuK0wA5ZxwL+6+11m9jEAd78euIPC1Ng1FKbH/nFGsaaqZUpHzUrjYptUD0Tli2i6EOdGponC3Z8HTq5w/PqStx34szTjSlvF0tHGHezc2wKlo2YlfbFNsgei8oW0CN0zO2NfumMlNz34AvsPHiodHd89edi9mI8/ujO/paM4JDnrqdp9nTumw5UvxPM5RHJA98wO2OLZU/mT33vLUGKY32qlozgkWaKoet+ELYUEpVf/IkoUWXvv4m7eu7g76zDaV7UeCDQ/syqv6z9EyozJOgCRTEX1OpqZWTXY+9j+MuCHeh9P3dL4xxTJiBKFtLfFF8P4iZWf65jW+MeNWv8hkjNKFCLjEpg9luf1HyJllChE9myt73gtqq3z0GI7ySElCpEkLupnfb6w3qNU2ovtnrqlMP33i1ML/6s/Ig1SohBJ4qK++GJ43zdgyhzACv+/7xvpzXpSM11ipOmxInGuoA5lSmyeN1OU4ChRiEA8i/pC2hJdzfT6hZLkA6TSk0hcQpoSG3ozPbT+iUp1kZQoROIS0qv4EJrp1YR4UQ4pyQdIiUIkLiG9is+6mR4lxItySEk+QOpRiMSlmS3Rk6iPh3q/hxAvyrrJVCSNKETi0uir+BBLMUkKaeQ1KORSXQB0PwqpTrNA0hF1T4wJE8P8/jfzu1E+OwwKF+WsS2Nt/vuu+1FI/UKa6tnqou6JsWdL8ZyAvv/N/m6Eeue/UEt1AchsRGFmc4DvAUcDA8AN7v71snPeDfwEGLzV2K3uPmrHSyOKGFR7lTtlDlz+TPrxtLJq3+tK0vz+V3uFrd+NlhTqiOIA8Ofu/piZTQYeNbN73X1F2Xm/cPcLMoivvYXYcGxVlZrg1aT1/Y8aNeh3o+1k1sx2943u/ljx7Z3ASmBWVvFImRAbjq2qUhO8Y3rlc9P6/kdNYdXvRtsJYtaTmc0HTgV+XeHpd5rZk2Z2p5mdEPExlplZv5n1b968OaFI24hmgaRr8cWFss0XtxX+P+/L2X7/o0YN+t1oO5knCjObBPwY+LS77yh7+jFgnrufDPwDsLzax3H3G9y9z937urq6kgu4XYS8YKsdZP39jxo1JB1baNt7SLbTY81sPPBT4G53//sazn8R6HP316LOUzNbpElZTWENdepsG4hqZmc2ojAzA24EVlZLEmZ2dPE8zGwJhXhfTy9KkRYW9co9qxFNiNt7SKaznk4HPgQ8bWZPFI99FpgL4O7XAx8APm5mB4A9wCXeiisERdJWy1qILNYVaEZVkDJLFO7+S8BGOec64Lp0IhJpI6He2Eh7LgUp82a2SCLUEI0W6it3zagKkhKFtJ5222SvEUmshYgjOWc920sq0l5P0npCLauEpJkt0SuJc28w7bkUHI0opPWEWlYJSdyv3DVbqaVpRCGtRw3R2sT5yl3JuaVpRCGtJ+6GqBrjo9P+Ty1NiUJaT5xlFTXGa6PZSi1NpSdpTXGVVdQYr02oNyOSWChRiERR7b12mq3UslR6EokSau1dfRNJkRKFSJQQa+/qm0jKlChEooS4UlhrFiRl6lGIjCa02rv6JpIyjShE8ibUvkmt1F/JHSUKyY9Wu8A0+vWE2DeplforuaREIfnQaheYZr6eEPsmtVJ/JZfUo5B8aLWFb81+PaH1TWql/kouaUQhI4VY4mm1C0yrfT21ynt/pU0pUchwoZZ4Wu0C0+jXE2ISr0ee+yttLNNEYWZLzWy1ma0xs6sqPH+Ymd1cfP7XZjY//SjbTBo15EYudq12gWnk6wk1idcjz/2VNpZZj8LMxgL/CJwNrAMeMbPb3X1FyWkfBba6+3FmdgnwZeCD6UfbRpIuiTR6J7RW23Suka+nVfo0ee2vtLEsm9lLgDXu/jyAmf0QuBAoTRQXAl8svv0j4DozM3f3NANtK0nf9KeZi12rXWDq/Xrata8hmcuy9DQLKL0irSseq3iOux8AtgNHVvpgZrbMzPrNrH/z5s0JhNsmki7x6GLXuFbr00huZJkorMKx8pFCLecUDrrf4O597t7X1dXVdHBtK+kasi52jWu1Po3kRpalp3XAnJLHs4ENVc5ZZ2bjgCnAlnTCa2NJlnjO+vzwHgXoYlerVuvTSG5kmSgeARaY2THAeuAS4I/KzrkduAz4FfAB4H71J3LsqVsO9ShsLPjBwohFF7vatVqfRnIhs0Th7gfM7BPA3cBY4CZ3f9bMrgb63f124EbgX8xsDYWRxCVZxStNKp/t5AcPjSR04RMJmrXiC/S+vj7v7+/POgwp9dUTq8ymmgOXP5N+PHk2ODJT+UliZGaPuntfpee015OkQ7Od4tHoOhSRJmgLD0mHZjvFQ7uvSgaUKCQdmtoZD43MJANKFJIO7fETD43MJAPqUUh6NLWzeVqHIhnQiEIkTzQykwxoRCGSNxqZSco0ohAZlPebAokkRCMKEdD6BJEIGlEM0qvJ9pbk+gT9bknOaUQBejUplbcXgebXJ+h3S1qARhSg1a7t7qlbqHzrE5pfn6DfLWkBShSg1a7t7r6rqXw/LGt+fYJ+t6QFKFGAVru2u6oXbW++PKTfLWkBShSgfYjaXdWL+ZzKx+uh3y1pAUoUoNWu7W60i3kzs5b0uyUtQDcuEoHqNwMqn7UEhSSii720GN24SFpDknd2q7YtRtSsJSUKaROZJAozuxZ4H7APeA74Y3ffVuG8F4GdwEHgQLVsJ20gq/UImrUkklmP4l7gRHdfDPwG+EzEuWe4+ylKEm0uq/UImrUkkk2icPd73P1A8eFDgP7qJFpWr+w1a0kkiFlPHwHurPKcA/eY2aNmtizqg5jZMjPrN7P+zZs3xx6kZCyrV/aatSSS3KwnM/s5cHSFpz7n7j8pnvM5oA94v1cIxMx63H2Dmc2gUK76H+7+wGifW7OeWpBmH4kkKpNZTzYArQYAAAVDSURBVO7+nqjnzewy4ALgrEpJovgxNhT/32RmtwFLgFEThbSgwWSQ1KwnEakqq1lPS4ErgXe5++4q50wExrj7zuLb5wDaSa2d6c5uIpnIqkdxHTAZuNfMnjCz66FQajKzO4rnzAR+aWZPAg8DP3P3u7IJV0SkfWUyonD346oc3wCcX3z7eeDkNOOSFpPkAj2RNqKV2dKadMMgkdiEMD1WJH66YZBIbJQopDVp6w2R2ChRSGvS1hsisVGikNakrTdEYqNEIa1JW2+IxEaznqR1aYGeSCw0ohARkUhKFCIiEkmJQkREIilRiIhIJCUKERGJlNiNi7JkZpuBtVnHUYejgNeyDqIBijs9eYwZFHfamol7nrt3VXqiJRNF3phZf7U7S4VMcacnjzGD4k5bUnGr9CQiIpGUKEREJJISRRhuyDqABinu9OQxZlDcaUskbvUoREQkkkYUIiISSYlCREQiKVEEwsz+ysyeMrMnzOweM+vJOqZamNm1ZraqGPttZjY165hGY2b/xcyeNbMBMwt+CqSZLTWz1Wa2xsyuyjqeWpjZTWa2ycyeyTqWepjZHDP7DzNbWfwd+VTWMY3GzA43s4fN7MlizP879s+hHkUYzKzT3XcU3/4k0OvuH8s4rFGZ2TnA/e5+wMy+DODuV2YcViQzWwQMAN8G/sLd+zMOqSozGwv8BjgbWAc8Alzq7isyDWwUZvb7wC7ge+5+Ytbx1MrMuoFud3/MzCYDjwIXhfz9NjMDJrr7LjMbD/wS+JS7PxTX59CIIhCDSaJoIpCLDO7u97j7geLDh4Dg7zXq7ivdfXXWcdRoCbDG3Z93933AD4ELM45pVO7+ALAl6zjq5e4b3f2x4ts7gZXArGyjiuYFu4oPxxf/xXr9UKIIiJn9jZm9DPxXII/37PwIcGfWQbSYWcDLJY/XEfiFq1WY2XzgVODX2UYyOjMba2ZPAJuAe9091piVKFJkZj83s2cq/LsQwN0/5+5zgO8Dn8g22kNGi7t4zueAAxRiz1wtMeeEVTiWi9FmnpnZJODHwKfLRvtBcveD7n4KhRH9EjOLtdynW6GmyN3fU+Op/wr8DPhCguHUbLS4zewy4ALgLA+k6VXH9zp064A5JY9nAxsyiqUtFOv8Pwa+7+63Zh1PPdx9m5n9P2ApENtEAo0oAmFmC0oe/gGwKqtY6mFmS4ErgT9w991Zx9OCHgEWmNkxZjYBuAS4PeOYWlaxMXwjsNLd/z7reGphZl2Dsw3NrAN4DzFfPzTrKRBm9mNgIYXZOGuBj7n7+myjGp2ZrQEOA14vHnoo9NlaZvaHwD8AXcA24Al3PzfbqKozs/OBrwFjgZvc/W8yDmlUZvYD4N0Utr1+FfiCu9+YaVA1MLPfBX4BPE3hbxHgs+5+R3ZRRTOzxcB3Kfx+jAFucferY/0cShQiIhJFpScREYmkRCEiIpGUKEREJJIShYiIRFKiEBGRSEoUIgko7kL6gplNLz6eVnw8z8zuMrNtZvbTrOMUqYUShUgC3P1l4FvANcVD1wA3uPta4FrgQ1nFJlIvJQqR5HwVeIeZfRr4XeArAO5+H7Azy8BE6qG9nkQS4u77zewK4C7gnOI24SK5oxGFSLLOAzYCubl5j0g5JQqRhJjZKRTuTPcO4PLi3dNEckeJQiQBxV1Iv0XhfgYvUWhg/122UYk0RolCJBl/Crzk7vcWH38TON7M3mVmvwD+DTjLzNaZWbA714qAdo8VEZFRaEQhIiKRlChERCSSEoWIiERSohARkUhKFCIiEkmJQkREIilRiIhIpP8PFOWsGNIKHbkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "utils.plotBestFit(train_X,train_y, coeff)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 随机梯度下降法\n",
    "\n",
    "随机梯度下降法求偏导的公式：  \n",
    "![](http://windmissing.github.io/images/2019/82.png)  \n",
    "推导过程见：https://windmising.gitbook.io/liu-yu-bo-play-with-machine-learning/6-1/6-6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "coeff =  [[ 1.01702007]\n",
      " [ 0.85914348]\n",
      " [-0.36579921]]\n"
     ]
    }
   ],
   "source": [
    "train_X,train_y=utils.loadDataSet()\n",
    "coeff = logRegres.stocGradAscent0(train_X,train_y)\n",
    "print (\"coeff = \", coeff)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXxU1fn48c9DEvYlLGHft6CyG1HrCsiirWvVitXa6ldqv9qqbanafr/W2m9/FRARxQ2XahettCq1LuzuihoEESqBsCfsAmEJS5bn90cGDWFmssyduefOPO/XixeZOzeZJ8nkPPec85xzRVUxxhhjIqnndwDGGGPcZonCGGNMVJYojDHGRGWJwhhjTFSWKIwxxkSV7ncA8dCmTRvt3r2732EYY0xgLF68eKeqZoV7LikTRffu3cnNzfU7DGOMCQwR2RDpORt6MsYYE5UlCmOMMVHFPVGIyDMisl1Ellc6do+IFIrI0tC/CyJ87lgRyRORfBG5M96xGmOMOV4iehTPAmPDHJ+qqoND/96o+qSIpAGPAOcDJwLjROTEuEZqjDHmOHFPFKr6LrCrDp86DMhX1bWqegT4O3Cxp8EZY4yplp9VT7eIyA+AXOAXqrq7yvOdgE2VHhcApyYqOGOCbtaSQibPyWPznoN0zGzEhDHZXDKkk99hmQDyazL7MaAXMBjYAkwJc46EORZxq1sRGS8iuSKSu2PHDm+iNCagZi0p5K6Xv6Bwz0EUKNxzkLte/oJZSwr9Ds0EkC+JQlW3qWqZqpYDT1IxzFRVAdCl0uPOwOYoX3OGquaoak5WVtg1I8akjMlz8jhYUnbMsYMlZUyek+dTRCbIfEkUItKh0sNLgeVhTvsU6CMiPUSkPnAV8Goi4jMm6DbvOVir48ZEk4jy2BeAj4BsESkQkRuASSLyhYgsA4YDt4fO7SgibwCoailwCzAH+BKYqaor4h2vMcmgY2ajWh03Jpq4T2ar6rgwh5+OcO5m4IJKj98AjiudNcZEN2FMNne9/MUxw0+NMtKYMCbbx6hMUCXlXk/GpLqj1U1W9WS8YInCGIfFUuJ6yZBOlhiMJyxRGOOooyWuR4ePjpa4ApYATELZpoDGOMpKXI0rrEdhjEMqDzVFWl1qJa4m0SxRGOOIqkNNkbhQ4mrbg6QWSxTGOCLcUFNVLpS42txJ6rE5CmMcEW1ISYBOmY3442UDfG+Mbe4k9ViPwhhHdMxsRGGYZNEpsxEf3DnCh4jCs+1BUo/1KIxxxIQx2TTKSDvmmAtDTVXZ9iCpxxKFMY64ZEgn/njZADplNnJqqKmqoCQ04x0bejLGIUFYTZ2I7UGsqsotliiMMbUWz4RmVVXusaEnY4xTrKrKPZYojDFOsaoq99jQkzEmLL/mCSKVCVtVlX+sR2GMOc7ReYLC0J5TR+cJZi0pjPtrh6uqAig+Uhrz689aUsgZ9y2kx52vc8Z9CxPy/SSDRNwK9RkR2S4iyysdmywiK0VkmYi8IiKZET53feiWqUtFJDfesRpjKvg5T3C0TDizUcYxx3cXl8SUrPxMfkGXiB7Fs8DYKsfmAf1VdSCwCrgryucPV9XBqpoTp/iMSWp1uYr2e57gkiGdaNLg+JHxWJKVTZLXXdwThaq+C+yqcmyuqpaGHi4COsc7DmNSUV2vol1Yfe11svI7+QWZC3MU1wNvRnhOgbkislhExicwJmOSwj2vrqjTVbQLq6+9TlYuJL+g8jVRiMhvgFLgbxFOOUNVhwLnAzeLyNlRvtZ4EckVkdwdO3bEIVoTbzbR6K1ZSwrZc7Ak7HPVXUW7sJ2I18nKheQXVKIa6T5aHr6ISHfgNVXtX+nYdcBNwEhVLa7B17gH2K+q91d3bk5Ojubm2tx3kIS7aU+jjDQn9zoKijPuWxi2zBQgs1EGTRqkO79FhtclurY1SGQisjjSXLAv6yhEZCxwB3BOpCQhIk2Aeqq6L/TxaODeBIZpEijaRKP9IddNtF7DgSOlX/c2XN4iw+utQoKwl5aLElEe+wLwEZAtIgUicgMwHWgGzAuVvj4eOrejiLwR+tR2wPsi8jnwCfC6qs6Od7zGHzbR6L1IY+/1BErKjh1JsOofE03cexSqOi7M4acjnLsZuCD08VpgUBxDMw4J2mrcIAxhTBiTHXY4L9LtVi0pm0hcqHoyJlATjUFZuBVpQrqTVf+YWrK9nowTEnGPg6NmLSnknldXfD1G37JxBr+98KQav1aQ5lMijcmH62n4kZSr9syG98virZU7nO6ppSJLFMYZiZhonLWkkAn/+JyS8m/G6HcXlzDhn59/HUN1gj6fksikHE24+078ddHGr593eZI91ViiMCll8py8Y5LEUSVlWuMeQdDmU8JxofonXM+sKld7aqnG5ihMUqjpYr1oV/017REEaT7FZTX9eQelp5bMLFGYwKvN5HK0q/6a9ghcWLWcDGr68w5STy1Z2dCTCbzaTC5PGJN93BwFQEaa1KpH4MLQTdCFK9+tynpqbrBEYQKvNpPLRxv3aFVPLq+RiDW2yp/folEGIrCnuMSX7zPcpLpVPbnJEoUJvNpOLkfrDYSrxHGl8ibW2Kp+fuUNA/36Pq1nFgw2R2ECz8vJZZdvbhNrbNVVGbnyfRr3WI/CBJ6X6wLisUbCq6GsWGOryXnRznF5SM7ElyUKkxS8GsLweo2El0NZscYW6fNr8rVcHpIz8WdDTybQvL7ZkddrJLwcyoo1tnCfX9Ov5fKQnIk/61GYwIrHVa7X21vEMlwUbqjnj5cNqHNsVb+32lQ9BX3bEhMbSxQmsOK1OV91VVG1aagzG2ewu/j425FWN1wUKQn+8bIBfHDniFp8N8eq6xBdMmxbYurOhp5MYCX6Kre224vPWlLI/kOlxx2vyeI+14Z6kmXbErsve91YojCBFelqNl5XubVtvCNtQNikfnq1V/WuDfUkw7YlQbmPiIsSkihE5BkR2S4iyysdayUi80Rkdej/lhE+97rQOatF5LpExGuCIdFXubVtvCMdLzp4/FBUVYlOgjVxyZBOfHDnCNbd920+uHNEoJIEuNdLC5JE9SieBcZWOXYnsEBV+wALQo+PISKtgN8CpwLDgN9GSigm9ST6Kre2jXcsjX2yDPW4xLVeWpAkZDJbVd8Vke5VDl8MnBv6+DngbeCOKueMAeap6i4AEZlHRcJ5IU6hmoBJ5BYQke5BHanxru35lSXq5kKptIjOJuTrzs+qp3aqugVAVbeISNsw53QCNlV6XBA6ZkzC1bbxjrWxr2kSrGtjn2qL6GJJ3KnO9fJYCXPs+NlBQETGA+MBunbtGs+YTAqrbQ8m3j2eWBr7oNz726tejyu3gI2HkrJyXvx0E+dmZ9G5ZWPPv76fiWKbiHQI9SY6ANvDnFPAN8NTAJ2pGKI6jqrOAGYA5OTkhE0mJn5SaQjDJZEa+1/M/JzbX1wa9XcRaWy+cM9BZi0pTMjvr7r3jde9nmTbrba8XHn9iy1MmZvH+q+K+eXovtwyoo/nr+NneeyrwNEqpuuAf4U5Zw4wWkRahiaxR4eOGYdY2aF/IjX2Zaox3e0vEb+/mrxvrFIpPFXl3VU7uOiR9/npC0tomJHGn354CjcP7x2X10tUeewLwEdAtogUiMgNwH3AKBFZDYwKPUZEckTkKYDQJPbvgU9D/+49OrFt3GF/zP6pyURsuN/FrCWFHDh8/GLAaJ/jtZq8byJtYpjKlUpLN+3h6ic/5gfPfMKe4hKmfm8Qb/zsLIb3a4tIuNH62CWq6mlchKdGhjk3F/ivSo+fAZ6JU2jGA1Z26J+a3E4Ujv1dVB3OqcnnxEN175tZSwoRwk9KpmKl0pod+7l/Th5vLt9K6yb1uefCExl3alcapEfe6NErrk9mmwCwskP/VJ2grSdCmR7ftFb+XVR3A6NwnxMP1b1vJs/JC5skBFKqUmlr0SEenL+KfywuoGF6PW47rw//dVZPmjZIXPNticLELFLZ4fB+WZxx38LATnC7MEFfkxgqT9CG6y1ULQGtSU8hEWWj1ZWrRopTSc7y3aqKikt49J18nv1gPeWq/OD0btwyvDetmzZIeCyWKEzMwpUdDu+XxUuLCwNbo+/CGoO6xFCTEtBIV/JpIpSrJiwpVhdrpDg7JXlP9eCRMv704Toef3sN+w6XcumQTtx+Xl+6tPK+7LWmRMN0U4MuJydHc3Nz/Q4jpZ1x38KIf+SxbJOdKC7EH68YIvU6XNvkLyhxeqWkrJx/5BYwbcEqtu09zMh+bZkwNpt+7Zsn5PVFZLGq5oR7znoUJi6CPsHtQvzxiiEoC8+CEmesVJU3vtjKlLl5rN15gJxuLZl+9VBO6d7K79C+ZonCxEXQJ7hdiL+mMdRlLiUoC8+CEmddvb96JxNnr+SLwiL6tmvKUz/IYeQJ8StzrSu7H4WJi6DvfupC/DWJwRY7BtMXBUVc89THXPP0x+w6cIQpVwzizVvP5rwT2zmXJMB6FCZOgj5s4EL8NYkhKPs1mQprd+xnytxVvP7FFlo1qc//fudErjktMWshYmGT2cYEWI87X4+41mDdfd9OdDgmgm17DzFtwWpe/HQTDdLr8V9n9eTGs3rQrGGGNy+wbCYsuBeKCqBFZxh5Nwy8slZfwiazjUlSns2leNDQmOMVHSzh8XfW8KcP1lFWrlx7WjduHt6brGYeroVYNhP+/TMoCb0PijZVPAbPfoeWKIwJME/usZCAhibVHCop49kP1/PY22vYe6iEiwd15OejsunaOg5rIRbc+83v7qiSgxXHLVEYYzyZS0lAQ5MqSsvK+efiAh6cv5qtew8xPDuLCWP6cWLHOK6FKCqo3fE6sERhTBwkcvuPmEtIE9DQJDtVZfbyrUyem8faHQcY2jWTaVcN5tSereP/4i06V/QCwx33iCUKYzzmwvYftZKAhiaZfZhfsRbi84Ii+rRtyoxrT2ZUIstcR9597NAhQEajiuMesURhjMcCV7KagIYmGS0vLGLi7JW8t3onHVs0ZPLlA7lsaGfS6iV4HcTR4cE4FiNYojDGYy5s/1ErCWhofOdhVdf6nQe4f24ery3bQmbjDP7n2ydwzWndaJjh41qIgVfG9fdlicIYj7mw/Uetxbmh8ZVHVV3b9x7ioYWr+fsnm8hIq8dPR/TmxrN70tyrtRAO8y1RiEg28GKlQz2Bu1X1wUrnnEvFvbTXhQ69rKr3JixIY+rAk5LVJODC/TyAmKu69h4q4Yl31vDM++spKStn3LCu/HRkb9o2axingN3jW6JQ1TxgMICIpAGFwCthTn1PVb+TyNiMiYUL23/4zakJ/TpWdR0qKeMvH23gkbfz2VNcwkWDOvKL0X3p1rpJHIJ0mytDTyOBNaq6we9AjPFCsu96Wh2nJvRrWdVVWlbOy58VMnX+KrYUHeKcvllMGJNN/04t4hyou1xJFFcBL0R47nQR+RzYDPxSVVckLixjTF04NaFfw6ouVWXOim3cPzeP/O37GdQlkweuHMzpvRKwFsJxvicKEakPXATcFebpz4BuqrpfRC4AZgF9Inyd8cB4gK5du8YpWuM3Z8a9TVROTejXoKpr0dqvmDh7JUs27qFXVhMev+Zkxpzk5pbffvB991gRuRi4WVVH1+Dc9UCOqu6Mdp7tHpucUu3WmJ5L4MZ/QfldrdhcxKTZebyzagftmzfk9lF9+O7QzqSnpd6telzfPXYcEYadRKQ9sE1VVUSGUXGjpa8SGZxxh1Pj3kGT4I3/XJ/Q3/hVMVPm5fGvpZtp0SiDX1/Qjx+c3t3ftRAO8zVRiEhjYBTw40rHbgJQ1ceBy4GfiEgpcBC4Sv3uAgVNEm0f7dS4t5/q8juNx8Z/1cTh4oT+jn2HeXjhap7/eCPpacJ/n9uLH5/TixaNkn8tRCx8TRSqWgy0rnLs8UofTwemJzqupJFk20c7Ne7tl7r+Tr3e+C9g7619h0p48t21PPX+Og6XlnPVKV24dWQf2jZPnbUQsUi9gbhUEu0qMoBcuI+17+r6O420wV9dN/4LyHvrUEkZT723lrMnvcVDC/MZ0a8t839+Dn+4dIAliVpwYY7CxEuSbR/t+rh3QtT1d+r1xn+Ov7fKypWXP6u4L0ThnoOc1acNvxrTjwGdU3ctRCwsUSSzJNw+OlHj3s6W4db1d+r1xn+OvrdUlflfbmfynJWs2rafQZ1bMPnygXyrdxtf4/paQOcMLVEkM9s+uk6c2n6iqlh+p15u/Ofge+uTdbuYOHslizfspmebJjz6/aGc37+9O2shAjavU5klimSWCttHx4HTZbiu/E5diQP4csteJs/JY+HK7bRr3oA/XjaAK052cC1EgG85a4ki2SXz9tFx4nwZriu/U5/j2LSrmAfmrWLW0kKaNUjnzvP7cd3p3WlU39G1EI7P60RjicI4w5V5AWfKcAM6nn2MOHwPO/cfZvrCfP728QbqifDjs3vxk3N60aKx42shHJ3XqQlLFMYJLs0LOHE/iQCPZ3/N4+9h36ESnnpvHU+9t5ZDpeVcmVOxFqJ9i4CUuTo4r1NTliiME1yaF3CiDLcu49mu9UA8GpM/XFrG3xZtZPpb+ew6cIRvD+jAz0f3pVdWU48DjjOH5nVqyxKFcYJr8wK+bz9R2/FsF3sgMY7Jl5Ur/1payAPzVlGw+yBn9G7Nr8b0Y1CXTA+DTDBX5pdqyRKFcYIz8wK1Ec8r+NqOZ7tYUdOoJRzcdfzxasbkVZWFK7czeU4eK7fuo3+n5vzxsgGc1ScrToGa6jhWP2ZSVeC25zh6BV+0CdBvruCXzfTm64+8u2L8urJo49muVdQsmwmH9x1/PK1+1DH53PW7uPKJj7jhuVwOlZQx/eohvHrzmZYkfGY9CuMEJ+YFaiPeV/C1Hc92raJmwb1QXnL88fpNw34PeVv3MXnOSuZ/uZ22zRrwh0v7c2VOFzJcWwuRoixRGGf4Pi9QG4m4gq/NeLZrFTWRfg4Hdx/zsGB3xVqIV5YU0rRBOhPGZHP9GT28XQvh2iR/AFmiMKYuXLuCd62ippqfz1f7D/PIW2v466INiMD4s3ryk3N7kdm4vrdxuDjJH0BRb4UqIs2BLFVdU+X4QFVdFu/g6spuhWrirmoDBBVX8Bc+ZA0QRPz5HBj7EE/tHsKT762l+EhpxVqI8/rQoUWciham9o+QsLrA7cvj85oBVadboYrIlcCDwHYRyQB+qKqfhp5+FhjqdaDGBIZrV/CuqfLzOdK8Gy90+S0Pv9mMnftXMfak9vxyTDa928Z5LYRrk/wBFW3o6dfAyaq6JXS/6r+IyK9V9WXAs+0YRWQ9sA8oA0qrZjSp2PpxGnABUExFwvrMq9c3ps4CWhOfMAOvpLz/Fbz6+WamzMtj0+KDnN6zGU9d14/BsayFqM2cQyxDhDa38bVoiSJdVbcAqOonIjIceE1EOgNe37d6uKrujPDc+UCf0L9TgcdC/xtjHKWqvJ23g4mzV7Jy6z5O6tic564fwNl92nyz7XddGuLazjnUdZLf5jaOES1R7BWRXkfnJ0I9i3OBWcBJiQgu5GLgz1oxmbJIRDJFpMPRJGaMccviDbuZOHsln6zbRbfWjXlo3BC+M6AD9epVGoioa0Nc27Lkug4RuriA0UfREsUdVBliUtV9IjIWuMvDGBSYKyIKPKGqM6o83wmo3HcsCB2zRGHcl0LDF6u37WPSnDzm/WcbbZo24PeX9Od7OV2onx5mLURdG+K6zDnUZYjQ5jaOES1RPAc8ISJTVLUUQETaAVOAbMCru6ifoaqbRaQtME9EVqrqu5WeDzcfctzQl4iMB8YDdO3a1aPQjBOC2timyPBF4Z6DTJ23ipc/K6BJ/XR+Obov15/Zg8b1ozQvdW2IE1WW7Fr5s8+iLXs8GegBLBGRESJyK/AJ8BEezhGo6ubQ/9uBV4BhVU4pALpUetwZ2Bzm68xQ1RxVzcnKsuX+SSPeW2XEU7Sr5iSw+8AR/u+1/zD8/rd59fPN3HBmD9791XBuGdEnepKAyA1udQ1xbbc2qatEvU5ARPxtqupu4KZQgphPReN8mqp61vcSkSZAvdCQVhNgNMf3VF4FbhGRv1ORoIpsfiKFBHmsOEmHL4qPlPL0e+uY8e5aDhwp5btDO3PbqL50qs0GjnWdZE5UWbKVPx8j2jqKTGAiFY3zWCrKU98UkVtVdaFHr98OeCVUBZEOPK+qs0XkJgBVfRx4I/Ta+VSUx/7Io9c2QZCIxjZeQ1tJNnxxpLScFz/dyLQF+ezcf5jRJ7Zjwphs+rRrVvsvFktDnKiyZCt//lq0/uFnwKPAzaE5irkiMhh4VEQ2qOq4WF9cVdcCg8Icf7zSxwrcHOtrmYCKd2Mbz3kE1/ZfqqPycuXfyzYzZe4qNu4qZliPVjxx7cmc3K1lbF/YGuLAiJYozq46zKSqS4FviciN8Q3LmJB4N7bxHNoK+PCFqvLu6p1Mmr2SFZv3ckKH5vzpR6dwbt+sb9ZCmJQQbY4iYt9eVZ+MTzjGVBHvxjbi0FaYXkxdBPSqecnGirUQi9buokurRky7ajAXDux47FoIkzJs91jjvng2tpGGtpCKYalYXjeAZb352/czec5K5qzYRpum9fndRScxbljX8GshTMqwRGFS28i74eXxHL80R2MbfgrYGootRQd5cN5q/rF4E43rp/PzUX254cweNGlgTYSxRGFS3cAr4eUIU26xVFYFpKx3T/ERHn17Dc9+uB4UfvitHtw8vBetmzbwOzTjEEsUxjRqBQd3hTkeQ1WP42soio+U8qcP1vP4O2vYf7iUy4Z05vZRfejcsrHfoRkHWaIwJh4cXUNRUlbOi59uYtqC1ezYd5jzTqhYC5Hdvg5rIUzKsERhTJX7OFd7vCZcWENRaTK9vHkXXu9zD1NWtmL9V8Wc0r0lj31/KDndWyUuHhNYliiMicfVv99rKCpNpr9X1p+JO8axfHtD+mXu55kfDmN4dltbC2FqzBKFMV5e/btSErvgXj4/3IFJpVfxQXl/Ost2Hsh4lIvrbySt3xeJj8cEmiUKY7y6+nekJHbNjv1M2XEZb5SfSmuK+G36c1ydtoAGUgp7rRcRkStJ3kGWKIwBbxb1+VwSu7XoENMWrGJmbgENdTC3pf+T/0p7g6Zy6JuTXNmQ0LVG2ZEk7ypLFMZ4xaeS2KLiEh59J59nP1hPuSrXntaNW9qvoM28N6GkUpJwZUNCFxvlgKx78YslCmO8kuCS2INHynj2w/U89nY++w6XcungTtw+qi9dWjUGToKGuHXVfpSLjbLj6178ZonCGK/EMilei6GY0rJyZuYWMG3BKrbtPczIfm355ZhsTujQ/NgTXd2Q0MVG2dF1L66wRGGMV+o6KV7DoRhV5c3lW7l/Th5rdx7g5G4teXjcUIb1CNhaCBcbZRfWvThMKu4LlFxycnI0NzfX7zCCz7UJx2Q1tX/4hrNRK6jfBIoK+KDhOUysdwPLdqXRt11TJozpx3kn+LgWIpb3RtXECBWN8oUP+T+hncLvdxFZrKo54Z7zrUchIl2APwPtgXJghqpOq3LOucC/gHWhQy+ranLcmd51Lk44JqtIQy4Hd/HFgRZMKr2D9w4NpJN8xf2nNeXSiy4gLRH3hYjUcMb63vB7MWK0uPyOwVF+Dj2VAr9Q1c9EpBmwWETmqep/qpz3nqp+x4f4UpuLE47JKsxQzLry9txfegWvl59OS/bxv+l/5vtpC2i4rj3Uuyz+MUVLBl68N6xRDhTfEoWqbgG2hD7eJyJfAp2AqonC+MHFCcdkVWl8fLtm8mDpZbxYNpwGlPCztJe5Mf11msnRBjtBP/9oycDeGynHiclsEekODAE+DvP06SLyObAZ+KWqrkhgaKnLxQnHZDXwSoqOwBNvfMwzxWdQRhrXNHifW3iBLNl77LmJ+vlHSwb23kg5vt/fUESaAi8Bt6lqlb8KPgO6qeog4GFgVpSvM15EckUkd8eOHfELOFWMvLtigrEyqwLx3KGSMp54Zw1nv57Jo8UjGDO4BwsmjOJ3380hq37JsScn8ucfqdE/Op8Qz/fGspkVE/z3ZFb8v2ymN1/X1JmvVU8ikgG8BsxR1QdqcP56IEdVd0Y7z6qePJLiVSDxVFpWzkufFfDg/NVsKTrEudlZTBiTzUkdW3xzkp8//+oqk+IVm6sVUSkgWtWTb4lCKur6ngN2qeptEc5pD2xTVRWRYcA/qehhRA3aEoVxlaoyZ8VWJs/JY82OAwzpmskdY/txWs/WiQ+musbej0QVqVS4RRe4fXl8XzvFOVkeC5wBXAt8ISJLQ8d+DXQFUNXHgcuBn4hIKXAQuKq6JGGMqz5cs5OJs/P4fNMeerdtyhPXnszoE9v5sxaiJiWuflQm2US5k/ysenofiPoXoqrTgemJiciY+FheWMSkOXm8u2oHHVs0ZNLlA/nu0M6JWQsRiavlzzZR7iQnqp6M8ZwD8yvrdx5gyrxV/PvzzWQ2zuB/vn0C15zWjYYZaQmNIyxXr9xtKw0nWaIwycfnVeXb9x3i4QX5vPDJRjLS6nHL8N6MP6cnzRtmxP21ayweV+5eJGdXV22nOEsUJvn4NKyy91AJM95Zy9Pvr6OkrJxxw7ry0xG9adu8Ydxes868vnL3Mjnbqm3nWKIwySfBwyqHSsr4y0cbeOTtfPYUl3DhoI78YlRfurdpEpfX84TXV+6uznkYT1iiMMknQROipWXlvLykkAfnrWJz0SHO7pvFr8Zk079Ti+o/2QVeXrm7OudhPGGJwiSfeAyrVLry1hF3Mzf9bCbPySN/+34Gdcnk/isH8a1ebbyJP4isWimpWaIwycfLYZUqY++Ldjdh4otbWVK+mJ5ZTXj8mqGMOam9f/eFcIVVKyU1SxQmOXk1rBIae/9PeVcmlV7F2+WDac9X3Nfsn1x+29Okp/m+XZobrFopqVmiMCaKjbsP80Dpf/Ov8m/RnGLuSn+e69Lm0LCkFNL+5F9gDqwTOY5VKyUtSxTGhLFj32GmL1zN80fuJ40yfpL2b36c/m9aSHHFCS26+Bec3X3QJJglCmMq2XeohCffXctT76/jcGk53+ut3Lrl17Qr2/LNSX6PvVspqkkwSxTGAIdLy/jroo088lY+uw4c4TPmDIgAABBHSURBVNsDO/DL0dn0aNMElh1xa5jHSlFNglmiMCmtrFx5ZUkhU+etonDPQc7q04ZfjenHgM6V1kK4NvYe9FJUF+dXTFSWKExweNjAqCrzv9zO5DkrWbVtPwM7t2DS5QM5o3cC10LU9fsJcimqza8EkiUKEwweNjCfrt/FxDdXkrthNz3bNOHR7w/l/P4JXgsRy/cT5FJUm18JJEsUJhg8aGBWbt3L5Nl5LFi5nXbNG/D/Lh3AFTmdyfBjLUSs349rw2E1ZfMrgWSJwhzPxTHkGBqYTbuKmTpvFa8sLaRZg3TuGNuPH36rO43q+3hfiFRtMIM+v5KifE0UIjIWmAakAU+p6n1Vnm8A/Bk4GfgK+J6qrk90nCnF1THkOjQwO/cfZvrCfP728QbqifDjs3vxk3N60aKxA/eFqGuD6WISr40gz6+kMN/2HxCRNOAR4HzgRGCciJxY5bQbgN2q2huYCkxMbJQpKNqQiFeWzYSp/eGezIr/l82s/nNG3l3RoFQWoYHZf7iUqfNWcc6kt/jLog1cfnJn3pkwnDvP7+dGkoBafT9fO5rEizYB+k0Sr8nPzxUDr4QLHwotWJSK/y98KFjJLgX52aMYBuSr6loAEfk7cDHwn0rnXAzcE/r4n8B0ERFV1UQGmlLiPSRS1x5LDSZwD5eW8fzHG5m+MJ+vDhzhggHt+cXobHplNfUmdi/VZUI6WSaCgzq/ksL8TBSdgMp97wLg1EjnqGqpiBQBrYGdCYkwFcV7DDmWxi5CA1NWrvxraSEPzFtFwe6DfKtXa+4Y249BXTK9iTleattgpuq8hvGdn4kiXC1i1Z5CTc6pOFFkPDAeoGvXrrFFlsriPYbsYWOnqryVt51Js/NYuXUf/Ts154+XDeDM3m2Sc9tvmwg2PvFzj+QCoPLOap2BzZHOEZF0oAWwK9wXU9UZqpqjqjlZWVlxCDdFxHsMOVKjVsvGbvGGXVz5xEdc/2wuh0rKmH71EF69+UzO6pOVnEkC6javYYwH/OxRfAr0EZEeQCFwFXB1lXNeBa4DPgIuBxba/EQCxGsMedlMOHLg+OO1aOzytu5j8pw85n+5jaxmDfi/S/rzvVO6+LMWItGCvNDOBJpviSI053ALMIeK8thnVHWFiNwL5Krqq8DTwF9EJJ+KnsRVfsVrYlR1EvuoRq3g/InVNnYFu4uZOm81Ly8poGmDdCaMyeZHZ3Sncf0UWwpkE8HGB77+lanqG8AbVY7dXenjQ8AViY7LxEG4SWyA+k2iNny7Dhxh+sJ8/rpoAwjceFZPfnJOL1o2qR/HYB0X9LUUJnBS7HLM+KaWk9gHDpfy9PvrmPHuWoqPlHLFyV249bw+dMxsFPb8lOHqgkiT1CxRmMSoYcXOkdJyXvhkIw8vXM3O/UcYc1I7JozJpnfbZgkK1HHJspbCBIolCpMY1ZTdlpcr/162mSlzV7FxVzGn9WzFkz/ox5CuLX0K2FG2lsL4wBKFSYwIFTs64AreDq2F+HLLXk7s0Jxnf3QK5/RN4jLXWNhaCuMDSxQmcapU7Hy2cTcTZyzi43W76NqqMdOuGsyFAztSr54liIhsUz3jA0sUJuHyt+9j0uw85v5nG22aNuD3F5/E907pSv30FFgLEStbS2F8YInCJMzmPQeZOm8VL31WQJP66fxiVF+uP7MHTRrY27BWbC2FSTD7CzVxt/vAER59O5/nPtoACtef0YP/Ht6bVq6thbD1CcaEZYniKGskPFd8pJRn3l/HE++s5cCRUr47tDO3jepLJxfXQtj6BGMiskQB1kh4rKSsnL9/spFpC/LZuf8wo06sWAvRt53DayHiuT7BLkJMwFmiAFvE5JHycuW1L7YwZW4eG74qZliPVjxx7cmc3C0AayHClZxC7OsT7CLEJAFLFGCLmGKkqry7eieTZq9kxea99GvfjD/98BTOzQ7IWohlM6m49UmYjYljXZ9gFyEmCViiAFvEFIMlG3czaXYeH639ii6tGvHg9wZz0aCArYVYcC/h74clsa9PsIsQkwQsUYAtYqqD/O37uX9OHrNXbKVN0/r87qKTGDcsoGshIjbaGvtVv12EmCRgiQJsEVMtbCk6yLT5q5mZu4lGGWncfl5fbjirB02DvBYiYmMeugFjLJPRdhFikoAk4w3jcnJyNDc31+8wksqe4iM89vYanv1wPapwzWnduHl4L1o3beB3aLELd1OljEYVt4CFyM/VNFlY1ZMJABFZrKo54Z4L8GWgSYSDR8p45oN1PP7OGvYfLuXSIZ24/by+dGnV2O/QvBOtRzm1f+yT0baS2gScL4lCRCYDFwJHgDXAj1R1T5jz1gP7gDKgNFK2M94rKStnZu4mps1fzfZ9hznvhLZMGNOP7PY+roWI55V5pMbcJqON8a1HMQ+4K3Tf7InAXcAdEc4drqo7ExdaaisvV95YvoUpc1exbucBcrq15JHvD+WU7q38Dcyv9Qg2GW0MvpSoqOpcVS0NPVwE2F+dA95fvZOLH/mAW55fQv20ejx9XQ7/uOl0/5MERF+PEE8j766Yk6jMJqNNinFhjuJ64MUIzykwV0QUeEJVZyQurNSxrGAPE2ev5IP8r+iU2YgHrhzExYM7kebSWgi/hoCsIs6Y+CUKEZkPtA/z1G9U9V+hc34DlAJ/i/BlzlDVzSLSFpgnIitV9d0IrzceGA/QtWvXmONPBWt37GfK3FW8/sUWWjWpz93fOZHvn9aVBulpfod2PD+HgGwy2qS4uCUKVT0v2vMich3wHWCkRqjRVdXNof+3i8grwDAgbKII9TZmQEV5bAyhJ71tew/xYGgtRMP0etw6sg83nt3T7bUQth7BGN/4VfU0lorJ63NUtTjCOU2Aeqq6L/TxaCDOA9LJrai4hMfeWcOzH66jrFy59rRu3DKiN22CsBbChoCM8Y1fl5DTgQZUDCcBLFLVm0SkI/CUql4AtANeCT2fDjyvqrN9ijfQDpWU8eyH63n0rXz2HS7lksGd+PmoAK6FsCEgY3zhS6JQ1d4Rjm8GLgh9vBYYlMi4kk1pWTn/WFzAg/NXsW3vYUb0a8uEMdmc0KG536Elhq2INsYTDg9Km7pSVd5cvpX75+SxducBhnbN5OFxQxnWw4Ey10Sx+0AY4xlLFEnmw/ydTJy9ks8LiujbrilP/iCH805oG4z7QnjJ7gNhjGcsUSSJ5YVFTJy9kvdW76Rji4ZMvnwglw3t7NZaiESyrTeM8YwlioBbt/MAU+bm8dqyLbRsnMH/fPsErjmtGw0zHFwLkUi29YYxnrFEEVDb9x5i2oLVvPjpJuqn1+NnI3pz49k9adYww+/Q3GDrLozxjCWKgCk6WMIT76zhmQ/WUVqmXH1qV346og9ZzQKwFiKRbN2FMZ6xRBEQh0rK+PNH63nkrTUUHSzh4sEd+fmovnRr3cTv0Nxl6y6M8YQlCseVlpXz0mcFPDh/NVuKDnFO3yx+NTabkzq28Ds0Y0yKsEThKFVlzopt3D83j/zt+xncJZMHrhzM6b1a+x2aMSbFWKJw0EdrvmLi7JUs3bSHXllNePyakxlzUrvUWwthjHGCJQqHLC8sYvKcPN5ZtYMOLRoy6bsDuWxoJ9LTfLm/lDHGAJYonLDhqwNMmbuKVz/fTGbjDH5zwQlce7qthTDGuMEShc8eXrCaaQtWk5FWj5uH92L82b1o0cjWQhhj3GGJwmedWzXiqmFd+NmIPrRt3tDvcIwx5jiWKHx26ZDOXDrEtpUwxrjLZkmNMcZEZYnCGGNMVL4kChG5R0QKRWRp6N8FEc4bKyJ5IpIvIncmOk5jjDH+zlFMVdX7Iz0pImnAI8AooAD4VEReVdX/JCpAY4wxbg89DQPyVXWtqh4B/g5c7HNMxhiTcvxMFLeIyDIReUZEWoZ5vhNQ+c4zBaFjxhhjEihuiUJE5ovI8jD/LgYeA3oBg4EtwJRwXyLMMY3yeuNFJFdEcnfs2OHJ92CMMSaOcxSqel5NzhORJ4HXwjxVAHSp9LgzsDnK680AZgDk5ORETCjGGGNqx5fJbBHpoKpbQg8vBZaHOe1ToI+I9AAKgauAq2vy9RcvXrxTRDZ4EmxitAF2+h1EHVjciRPEmMHiTrRY4u4W6Qm/qp4michgKoaS1gM/BhCRjsBTqnqBqpaKyC3AHCANeEZVV9Tki6tqVnzCjg8RyVXVHL/jqC2LO3GCGDNY3IkWr7h9SRSqem2E45uBCyo9fgN4I1FxGWOMOZ7L5bHGGGMcYInCDTP8DqCOLO7ECWLMYHEnWlziFlUrEDLGGBOZ9SiMMcZEZYnCGGNMVJYoHCEivw9tabJUROaGSoWdJyKTRWRlKPZXRCTT75iqIyJXiMgKESkXEedLIIO4i3Joa57tIhJujZSzRKSLiLwlIl+G3iO3+h1TdUSkoYh8IiKfh2L+neevYXMUbhCR5qq6N/Txz4ATVfUmn8OqloiMBhaG1r1MBFDVO3wOKyoROQEoB54AfqmquT6HFFFoF+VVVNpFGRjn+i7KInI2sB/4s6r29zuemhKRDkAHVf1MRJoBi4FLXP55i4gATVR1v4hkAO8Dt6rqIq9ew3oUjjiaJEKaEGVfK5eo6lxVLQ09XETFVitOU9UvVTXP7zhqKJC7KKvqu8Auv+OoLVXdoqqfhT7eB3yJ45uRaoX9oYcZoX+eth+WKBwiIn8QkU3A94G7/Y6nDq4H3vQ7iCRjuyj7RES6A0OAj/2NpHoikiYiS4HtwDxV9TRmSxQJVM2Ouqjqb1S1C/A34BZ/o/1GdXGHzvkNUEpF7L6rScwBUatdlI03RKQp8BJwW5XevpNUtUxVB1PRox8mIp4O9/l5h7uUU9MddYHngdeB38YxnBqrLm4RuQ74DjBSHZn0qsXP2nW12kXZxC40zv8S8DdVfdnveGpDVfeIyNvAWMJvtlon1qNwhIj0qfTwImClX7HUhoiMBe4ALlLVYr/jSUJf76IsIvWp2EX5VZ9jSlqhieGngS9V9QG/46kJEck6Wm0oIo2A8/C4/bCqJ0eIyEtANhXVOBuAm1S10N+oqici+UAD4KvQoUWuV2uJyKXAw0AWsAdYqqpj/I0qMhG5AHiQb3ZR/oPPIVVLRF4AzqVi2+ttwG9V9Wlfg6oBETkTeA/4goq/RYBfhzYodZKIDASeo+L9UQ+Yqar3evoaliiMMcZEY0NPxhhjorJEYYwxJipLFMYYY6KyRGGMMSYqSxTGGGOiskRhTByEdiFdJyKtQo9bhh53E5HZIrJHRF7zO05jasIShTFxoKqbgMeA+0KH7gNmqOoGYDJwrV+xGVNbliiMiZ+pwGkichtwJjAFQFUXAPv8DMyY2rC9noyJE1UtEZEJwGxgdGibcGMCx3oUxsTX+cAWIDA37zGmKksUxsSJiAym4s50pwG3h+6eZkzgWKIwJg5Cu5A+RsX9DDZSMYF9v79RGVM3liiMiY8bgY2qOi/0+FGgn4icIyLvAf8ARopIgYg4u3OtMWC7xxpjjKmG9SiMMcZEZYnCGGNMVJYojDHGRGWJwhhjTFSWKIwxxkRlicIYY0xUliiMMcZE9f8BRGnZ5SsTLMcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "utils.plotBestFit(train_X,train_y, coeff)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 学习率衰减的随机梯度下降法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "coeff =  [[14.59044183]\n",
      " [ 1.21558376]\n",
      " [-1.99701652]]\n"
     ]
    }
   ],
   "source": [
    "train_X,train_y=utils.loadDataSet()\n",
    "coeff = logRegres.stocGradAscent1(train_X,train_y)\n",
    "print (\"coeff = \", coeff)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de3Sc9X3n8ffXsmyksfFtBPiK5EKdBkIgVUlScnKBAg4lgWS7lOyeLttm601Ps03YlhOS7CYsbU9o2TS90IbShSbtpkloAw5NSIBC9yThhAYB5pIQWmobfKFg+W5JtiX5u3/MSB6NZkZzeS6/Z+bzOkdHmmceSd/R5fk+v9/3dzF3R0REpJp5aQcgIiJhU6IQEZGalChERKQmJQoREalJiUJERGqan3YAccjn897f3592GCIimfHEE08Mu3tfpefaMlH09/czNDSUdhgiIplhZi9Ve05dTyIiUpMShYiI1BR7ojCzu8zsNTN7ruTYTWa2y8y2FN+uqPK5G83sBTN70cxujDtWERGZLYkWxReAjRWOf87dzy++3V/+pJl1AX8KvBt4PfABM3t9rJGKiMgssScKd/8OsK+JT70QeNHdt7r7ceArwFWRBiciInNKc9TTh83sPwFDwG+6+/6y51cDO0oe7wTeXO2LmdkmYBPAunXrIg5VJHs2P7WLWx94gd0Hxli1tIcbLt/A1ResTjssyaC0itmfB34COB94BfhshXOswrGqS926+x3uPujug319FYcCi3SMzU/t4uP3PMuuA2M4sOvAGB+/51k2P7Ur7dAkg1JJFO7+qrtPuvsJ4C8odDOV2wmsLXm8BtidRHwiWXfrAy8wNj4549jY+CS3PvBCShFJlqWSKMxsZcnD9wHPVTjtceBsMxswswXAtcB9ScQnknW7D4w1dFykliSGx34Z+D6wwcx2mtkHgd83s2fN7BngXcD1xXNXmdn9AO4+AXwYeAB4Hrjb3X8Yd7wi7WDV0p6GjovUEnsx290/UOHwnVXO3Q1cUfL4fmDW0FkRqe2Gyzfw8XuendH91NPdxQ2Xb0gxKsmqtlzrSaTTTY1u0qgniYIShUibuvqC1UoMEgklCpGAaS6EhECJQiRQU3MhpuoMU3MhACULSZRWjxUJlOZCSCjUohAJSGlXU7VlCEKYC6Eusc6iRCESiPKupmrSnguhLrHOo64nkUBU6moqF8JcCHWJdR61KEQCUatLySCYLh4tD9J5lChEArFqaQ+7KlxsVy/t4dEbL04hosqqxZl2l5jER11PIoG44fIN9HR3zTgWQldTuazEKdFRi0IkEFlZdiOJODWqKizmXnUvoMwaHBz0oaGhtMMQkSZUGv3V093FZ97/BiWLGJnZE+4+WOk5dT2JSFA0qio8ShQiEhSNqgqPEoWIBEWbLoVHiUJEKtr81C4uuuURBm78Jhfd8gibn9qVyPfVqKrwaNSTiMyS5jIdcY6q0miq5qhFISKzpF1QvvqC1Tx648V87hfPB+D6r25puVUzlfx2FRdcnEp+SbWUsiz2RGFmd5nZa2b2XMmxW83sx2b2jJnda2ZLq3zudjN71sy2mJnGu4okJISCctQX9rSTX5Yl0aL4ArCx7NhDwLnufh7wz8DHa3z+u9z9/Grje0WktmZqDSEUlKO+sIeQ/LIq9kTh7t8B9pUde9DdJ4oPHwPWxB2HSCf6H5uf5fqvbmn4rjyEgnLUF/YQkl9WhVCj+BXgW1Wec+BBM3vCzDbV+iJmtsnMhsxsaM+ePZEHKZI1m5/axZcee3nWBkj13JVffcFqPvP+N7B6aQ9GYWHCpGdGR31hDyH5ZVWqo57M7JPABPClKqdc5O67zew04CEz+3GxhTKLu98B3AGFJTxiCVhipREp0br1gRda2iXv6gtWp/rzv+HyDRWX8mj2wp6VtbRClFqiMLPrgCuBS7zKglPuvrv4/jUzuxe4EKiYKCTbtGta9GolgyU93Vx0yyNBXzDjuLCnnfyyKpVEYWYbgY8B73D30Srn5IB57n64+PFlwM0JhikJqlW41D92c6rtGwEwcnyCA2PjQNhJWRf2MCQxPPbLwPeBDWa208w+CNwGLKbQnbTFzG4vnrvKzO4vfurpwPfM7GngB8A33f3bcccr6dCIlOhV6pM3ILegi/HJmY14DROVWmJvUbj7ByocvrPKubuBK4ofbwXeGGNoEpCs7ZqWhXpKta6b67+6peL5SspSjZbwkCBEXbiMU5bqKZW6bm594IVMJWVJnxKFBCHJESmbn9rFTff9cLqPfllvN59+zzl1f6+s11NCSsrlLbN3va6Pf/zxnqBbap1IiUKCkUThcvNTu7jhb59m/MTJPvr9o+Pc8HdPT8cwl6zXU0IZJlqpZfZ/H3t5+vmQW2qdRolCOsqtD7wwI0lMGZ/0ulsEWaunVBLCaKJKLbNyWWqptbMQZmaLtKze9Yxq3fXX2yLQDN9o1PvzzkpLrZ0pUUjmNbLKaK27/npbBCEsb9EO6v15Z6ml1q7U9SSZ10hx+YbLN8yqUQB0d1lDLYIQum6yrlJRvZxaamFQopDMa6S4PHVxrzXqKeQ5Eq3GFtJrq1RU16inMFmVZZYybXBw0IeGtM9Rp7jolkcqFpdXL+3h0RsvbuhrlY/EgcJdbQhdS63GVunzjcISzat1Ue54ZvZEtX1/VKOQzIuyuBzyLmitxlbp86duE7UtqNSiRCGZF2VxOeQ5Eq3GNtd5oSRECY9qFNIWoiouxzFHIqq6QKux1VpNdkqtZBJSfUOSpRaFZFoz+0HXEvUciUaG7sYdW6XPL1ct6UT5OiR7lCgks+K4eEU9RyLKmkersZV+PhQK2aVqJZ2QazcSP3U9SWbFtThflHMkWqkrVOvqieq1NdKVFHLtRuKnRCGZlcbFq9F++qW93ewfHZ91fK66QhJLmTeSdNphfStQnaVZ6nqSzKp2kYrr4tVoV9fmp3Zx5OjErOP1zAIPraunHda3Up2leUoUkllJX7wavXhXW6k2t2D+nHexoXX1tMP6VqEl3yxJpOvJzO4CrgRec/dzi8eWA18F+oHtwDXuvr/C514H/I/iw99x9y8mEbOEL+l9FRq9eFc7fnBsdldUuRC7erK+vlVoyTdLkmpRfAHYWHbsRuBhdz8beLj4eIZiMvk08GbgQuDTZrYs3lAlS66+YDWP3ngx2275eR698eJYL2SNdnW10jXWDl09oUm6q7KdJJIo3P07wL6yw1cBU62DLwJXV/jUy4GH3H1fsbXxELMTjkgiGr14t3KxT6qrJ+p5KCFT8m1emqOeTnf3VwDc/RUzO63COauBHSWPdxaPiSSu0a6uVrvG6u3qaXYkTxIjq0ISyhawWZTY6rFm1g98o6RGccDdl5Y8v9/dl5V9zg3AQnf/neLj/wmMuvtnK3z9TcAmgHXr1v30Sy+9FNdLEQlGKyvKRrnqrmRfqKvHvmpmKwGK71+rcM5OYG3J4zXA7kpfzN3vcPdBdx/s6+uLPFiRELUykqdaEXfXgbHEuqDq6frqpO6xUKWZKO4Drit+fB3w9QrnPABcZmbLikXsy4rHJDD6Z05HrYv9XL+LWkXcJOYX1DOvQXMfwpBIojCzLwPfBzaY2U4z+yBwC3Cpmf0LcGnxMWY2aGb/B8Dd9wG/DTxefLu5eEwCon/m9NS62M/1u6i1SGAS8wvqaQ1p7kMYkhr19AF3X+nu3e6+xt3vdPe97n6Ju59dfL+veO6Qu/+Xks+9y93PKr79ZRLxSmP0z5yeelaErfS7mCqA19qvOu75BfXMa6i2LLrmPiRLaz1JyzSRKT3lI3mqDU0p/V1UKoBXEvf8grkmFW5+atf0Vq1JxyYzKVFIy6r9wy/t7eaiWx7J7FDErCwgVzqMttpIptIL61wtCUhmfsENl2+oOGJr6vve+sALFZOEFT9XkqO1nqRllbo/uruMI0cnMlu3CKXu0ugggXomldVq6SW5jtNckwqrxem05zyPkKlFIS2rNJFp5NgEB8rWNIpir4ikxLXXRSOamRBXz6Syai3ANOZP1JpUWCtOgUNHx9k+PMK2krf3XbCad26oNHe5NUoUEonyf/iBG79Z8bys1C1CqLs0m6zmmtE9V5dPKLISZ5yOjk+yfe8I2/aMsK34fvveQlIYPnJ8+jwzWLWkh7edlY8lDiUKiUWIq582IoT440pWWVnKIitxtmp88gQ79o3OaBlsGx5h+/AIuw8enXFu3+KFDORzXPK60xnoy9G/Isf6vhzrlvdyyhyj31qhRCGxyPrdYAjx15usmim6Z2XJ8KzEOZcTJ5zdB8emE8DWkmSwY/8YkyX7lizp6WYgn+PN61cwkM9Nv/XncyxamM4lW4lCYpH1u8EQ4q8nWXXawn4hc3f2HDk23T20dfhkV9H2vaMcnzgxfW5PdxcD+RznrFrCleetmk4E6/M5luUWpPgqKktsUcAkDQ4O+tDQUNphiLRsrtaCFvZL3oHR4zO6h7YOj0zXEUaOn0zqC7rmsW5F73T30ED+ZFfRaYsXYmYpvorZai0KqBaFSMDm6noJoejejkaPT8xOBsXH+0dPjuabZ7BmWS8D+RyDZy6f0TJYtbSHrnkJJYNn7oaHb4aDO2HJGrjkU3DeNZF9eSUKkQyLrOge84UmRMcmJtmxb5StJSOJpt5ePXRsxrlnnHoKA/kcG89dyfpiMhjI51i7vIeF8+MrItflmbvh738Dxot/Bwd3FB5DZL9DJQqRDIuk6J7AhSYtkyecXfvH2Dp85OScg72jbBs+wq79Y5TUkFmeW8BAPsfbzupjfXFEUaGF0EvvgoAvlQ/ffPJ3N2V8rHBciUJEIim6J3ChiZO78+qhY8VkUEgC24rvX943yvjkyWywaOF8BvI5zl+7jPddsIaBfC8D+UUMrMixpLc7xVfRgoM7GzveBCUKkRgkuU5Uy0NIE7jQtMrd2T86PiMJbB8ena4dlLaoFsyfx8CKHGedtohLX3/GyWSQz5FftCC4InLLlqwptAIrHY+IEoVIxDI3ZDWBC029Dh8dL7QKSmYhF4aZHuHQ0Ynp87rmGeuW99K/ope3rl/BQF+OgRU5BvpyrDz1FOYlVUQOwSWfmtl1CNDdUzgeESUKkYiFsE5UQxK40JQ6Oj7JS3tLZyKfbB0MH5lZRF69tIf+fC/vPX9VsVVQaB2sWdZDd5fWNAVOdg9q1JNIdmRuyGoMF5rxyRPs3D82q6to2/AIuw+OUTp9K79oIQP5Xi5+Xd+MZHDminiXpWgr510Taz1JiUIkYiGsE9WwJi40J044rxw6enJJipJhpjv2jTJRMqTo1FPmM9C3iJ/pX8ZAfi39+V7W5xfRn+9l8SkJFJE7cPhvlJQoRCIWwjpRUXF3ho8crzjxbPveEY6VLEtxSvc8BvKL+KmVi7niDWewb+Q4D/3oVYaPHGfRwvn88s/2p9P11sbDf5OSWqIwsw3AV0sOrQc+5e5/WHLOO4GvA9uKh+5x95sTC1KkCSGsE9Wog2PjlZPB8AiHj50sInd3FYrIA/kcb//J/PTEs4F8jtMXnywilxf0dx88ml5BP+PDf0OQWqJw9xeA8wHMrAvYBdxb4dTvuvuVScYm0qoQVz0dPT7B9uHRWbOQtw2PsG9k5t4Ga5b10L8ix/vftHo6GazPL2LV0lOYX0cROaiCfgaG/4YulK6nS4B/dfeX0g5EJMuOT5zg5X2jJbOQi5veDI/wb4dm7m1w+qkL6V+R4/JzTp+ehby+L8fa5b0tL0sRVEE/oOG/WRVKorgW+HKV595qZk8Du4HfcvcfVjrJzDYBmwDWrVsXS5AiIZg84ew+MDarVbBteISd+0dnLEuxrLeb/nyOnz1rxfQ8g6lVTHMx7m0QVEE/4eG/7Sj1ZcbNbAGFJHCOu79a9typwAl3P2JmVwB/5O5nz/U1tcx4+0pyxnOa3J3XDh+rmAxe3jvK8cmTReTcgq6S7qHcjLrB0t509jYor1FAoaD/mfe/Ib2CtkY91RT6MuPvBp4sTxIA7n6o5OP7zezPzCzv7sOJRihByNyM5zrsHzk+o3uodEby6PGZy1KcubyX9fkcl/zUaYXWQTEZ9NW7t0GCF8vgCvoxzzNodyEkig9QpdvJzM4AXnV3N7MLgXnA3iSDk3AEVSBtwJFjEydrBmWb3Rwo2duga56xdlkP/fkcb16//OQWmCsi2NsgjiGicySeEAv60pxUE4WZ9QKXAv+15NiHANz9duAXgF8zswlgDLjW0+4ry5o2anIHVSAtc3R8kpf3jc5OBsMjvHZ45rIUK5cU9jb4+TesnLEf8tplvSyYX8eyFM38TqMeIqq5CR0l1UTh7qPAirJjt5d8fBtwW9JxtY02+2dOu0A6MbUsxd6Zs5C3DY+w60D5shQL6F+R4x0/2Te941l/sXXQs6CFEUXN/k6jHiKquQkdJYSuJ4lLm/0zJzHj+cQJ599KlqUoHWb68t6Zy1IsXjifgb4cP33mMv7dm9ZMb3bTn8+xpCemZSma/Z1GPURUcxM6ihJFO2uzf+aoCqTuzt6R47OTQbFucHR85rIU/StybDh9MZefc8aMkUUrcinsbdDs7zTqIaKam9BRlCjaWRv+MzdSID10dHxGEih9O1yyt8H8eSeXpbjorDwD+Ry7D4zxtSd38uqhYxw+OsHl55wRRmG22d9p1CvEam5CczJaM1SiaGcd8M98dHyyUCvYM3MW8va9IwwfmbksxaolPazvy/G+C1YXZiIXN7tZs6xnxrIUm5/axV8+uj3MYbit/E6jHCKawB4IbSfDNcPUJ9zFQRPuSmT0DqbU+OQJduwbndUq2D48wu6DM5el6Fu8sOLEs3XL69/b4KJbHqlYNF+9tIdHb7w4ktfUkjb4nXakz51bpTW4Fq5/Lvl4yoQ+4U7ilJGJRlPLUpQvWLd9eIQd+8eYLCkiL+npZiCf4y3rV8xIBv35HIsiWJYi5GG4QGZ+p1ImwzVDJQpJjLuz58ixirOQt+8d5XjJ3ga9C7roX5HjnNVLuPK8VdOJYH0+x7JcvMtSpD0Md1o7tBza4TVEJcM1QyUKidyB0eMVZyFv2zPCSOmyFF3zWLeiUER+54bTODg6zsPPv8rwyHGW9nSz6e3rU6kJBLHxUIb7s6e1w2uIUoZrhkoU0pSRYxPT3UTlw0z3lyxLMc9gzbJCMhg8c/mMlkHpshQhbXQTxDpFzcyXCO3uvc3m8bQswwMAlCikqmMTk+zYN8rWklnIUx+/emjmshRnnFpYlmLjuStZX1IzWLe8vmUpQlvHKfV1ihrtzw7x7j3DffKxyWh9qWaiKC7z3efu/1p2/Dx3fybWyCQRkyecXfvH2Dp8ZLpFMNVVtGv/2Iy9DZbnFjCQz/G2s/pYX7KvQX++l94Frd1zBF9AriTOO/hG+7NDvHvvWQZj+2Yfz0CfvMxU9b/bzK4B/hB4zcy6gf/s7o8Xn/4C8Kb4w5MouDuvHjrG1uEj011F03sb7BtlfPJkNli0cD4D+Rznr13G+y5Yc3KY6YocS3pjWpaCgArI9Yr7Dr7R/uzQ7t6fuRuOHZ59vGtBJvrkZaZat4GfAH7a3V8pLvH912b2CXe/B0h43QKZi7uzf3ScbcNHZnQVbRsubItZ2q2zYP48BlbkOOu0RVz6+jNmzDnIL0phWQoCKSA3Iu47+Eb7s0MbUfPwzXBifPbxBYsy2fXS6Wolivnu/gqAu//AzN4FfMPM1gDtN0svIw4fHWf78Gixq2iUbcNH2LZ3lG17jnCoZFmKrpJlKd66fsX0LOSBvhwrTz2Fea3sbRCDIArIjUjiDr6R/uzQRtRU+zmM7U82DolErURxyMx+Yqo+UWxZvBPYDJyTRHCd6uj4JC/tLSaBYjIoJIcRho+cLCJPLUsxkM/x3vNXMZBfxEC+l4H8ItYs66G7a+4ickhSLyA3IrQ7+NBG1IT08wltNFgG1UoUH6Osi8ndD5vZRuDjsUbVQbbsOMATL+2fTgbbhkfYfbB8b4OFrM/nuPh1fTOSwZkr6l+WQiIW2h08hDWiJpSfT4ijwTKoVqL4IvDnZvZZd58AMLPTgc8CG4CbE4iv7d375E6++P2XOPWU+Qz0LeJn+pcxkF9Lf76X9flF9Od7WXxKfEVkaVJod/ChCeXnE+JosAyquiigmS0DPgNcBHwEeAPw34HfBz7v7icqfmIAsrQo4GuHjzJ/3jyW9XanUkQWaWs3LaVySdXgpgNJRxO0phYFdPf9wIfM7CPAPwC7gbe4ewfPloneaYtPSTsEkWxppObQSq1EtY1pVaudZrbUzP4c+GVgI/B3wLfMLNJ1ls1su5k9a2ZbzGxWM8AK/tjMXjSzZ8xM8zdE2sEzdxeW3r5paeH9M3fX9zl//xvFi7+frDlU+9xLPlWojZSqp1bS6Pdpc7WGxTwJ/Asw6O4PuvtHgV8CfsfMvhxxHO9y9/OrNHveDZxdfNsEfD7i7y0iSWv2Qlyr5lDJedfAe/64sOcDVnj/nj+eu2XQ6Pdpc7WK2W8v72Zy9y3Az5rZr8Yb1gxXAX/lhWLKY8WWzsqpOR4iQVP3RWXNFpmbmb/SzGiw0Ga6p6xqi6JWLcLd/yLCGBx40MyeMLNNFZ5fDZR2Mu4sHpvBzDaZ2ZCZDe3ZsyfC8CR1zXRRhEDdF9U1eyGuVluIen5GUt8nI0KYkXWRu7+JQhfTr5vZ28uerzQUaNYwBne/w90H3X2wr68vjjglDVm+2Kr7orpmL8TN1hwaldT3yYjUE4W77y6+fw24F7iw7JSdwNqSx2sojMCSTpDli626L6pr9kLcbM2hUUl9n4xIdT8KM8sB84ozvnPAZcyeyHcf8GEz+wrwZuCg6hMdJMsX25CWsQhNKxPykpqBHtJM95SlvXHR6cC9xYlm84G/cfdvm9mHANz9duB+4ArgRWCUwnBd6RRJXGzjKjiHsoxFqHQhzoxUE4W7bwXeWOH47SUfO/DrScYlAYn7YhvnWkChLGMh0qKqS3hkWZaW8JA6xDnE9HPnVm6x9CyHj22L5nuIZEBTS3iIBCPOLoqq+ybsKyQo3f2LpD/qSSRVtWodrY6syur8D5EyShTS2WrVOloZWZXl+R8iZZQopLOddw105yo/17Os+a+b5fkfImWUKETmL4z+a2Z5/odIGSUKkbH9jR2vh9YKkjaiRCESx0U9hLWCVEyXiChRiMRxUU97rSAV0yVCmkchEuUM6lD2n2h2vweRCpQoRCCaSX1xLgfSKBXTGxdKkg+Qup5EohLSkNjQi+mh1U/UVVeTEoVIVEK6iw+hmF5NiBflkJJ8gJQoRKIS0l182sX0WkK8KIeU5AOkGoVIVFpZEj2O/vFQ93sI8aKsTaZqUotCJCrN3sWH2BUTp5BaXlNC7qoLgPajkOo0CiQZtfbEWJAL8+ffyt9G+egwKFyU0+4a6/C/d+1HIY0Laahnu6u1J8bYvuI5Af38W/3bCHXnv1C76gKQWovCzNYCfwWcAZwA7nD3Pyo7553A14Gprcbucfc5K15qUUSg2l3ukrVw/XPJx9POqv2sK0ny51/tDlt/G20p1BbFBPCb7v6kmS0GnjCzh9z9R2Xnfdfdr0whvs4WYsGxXVUqgleT1M+/VqtBfxsdJ7Vitru/4u5PFj8+DDwPrE4rHikTYsGxXVUqgvcsr3xuUj//WkNY9bfRcYIY9WRm/cAFwD9VePqtZva0mX3LzM6p8TU2mdmQmQ3t2bMnpkg7iEaBJOu8awrdNjcdKLx/9++l+/Ov1WrQ30bHST1RmNki4GvAR939UNnTTwJnuvsbgT8BNlf7Ou5+h7sPuvtgX19ffAF3ipAnbHWCtH/+tVoNcccW2vIeku7wWDPrBr4BPODuf1DH+duBQXcfrnWeitkiLUprCGuoQ2c7QK1idmotCjMz4E7g+WpJwszOKJ6HmV1IId69yUUp0sZq3bmn1aIJcXkPSXXU00XALwHPmtmW4rFPAOsA3P124BeAXzOzCWAMuNbbcYagSNLqmQuRxrwCjagKUmqJwt2/B9gc59wG3JZMRCIdJNSNjbTmUpBSL2aLxEIF0dpCvXPXiKogKVFI++m0RfaaEcdciCiSc9qjvaQirfUk7SfUbpWQtLIkeiVRrg2mNZeCoxaFtJ9Qu1VCEvWdu0YrtTW1KKT9qCBanyjv3JWc25paFNJ+oi6IqjA+N63/1NaUKKT9RNmtosJ4fTRaqa2p60naU1TdKiqM1yfUzYgkEkoUIrWo771+Gq3UttT1JFJLqH3vqptIgpQoRGoJse9ddRNJmBKFSC0hzhTWnAVJmGoUInMJre9ddRNJmFoUIlkTat2kXqqvZI4ShWRHu11gmn09IdZN6qX6SiYpUUg2tNsFppXXE2LdpF6qr2SSahSSDe028a3V1xNa3aReqq9kkloUMluIXTztdoFpt9dTr6zXVzqUEoXMFGoXT7tdYJp9PSEm8UZkub7SwVJNFGa20cxeMLMXzezGCs8vNLOvFp//JzPrTz7KDpNEH3IzF7t2u8A083pCTeKNyHJ9pYOlVqMwsy7gT4FLgZ3A42Z2n7v/qOS0DwL73f0sM7sW+D3gF5OPtoPE3SXS7E5o7bboXDOvp13qNFmtr3SwNIvZFwIvuvtWADP7CnAVUJoorgJuKn78d8BtZmbu7kkG2lHi3vSnlYtdu11gGn09nVrXkNSl2fW0Gii9Iu0sHqt4jrtPAAeBFZW+mJltMrMhMxvas2dPDOF2iLi7eHSxa1671WkkM9JMFFbhWHlLoZ5zCgfd73D3QXcf7Ovrazm4jhV3H7Iuds1rtzqNZEaaXU87gbUlj9cAu6ucs9PM5gNLgH3JhNfB4uziueRTM2sUoItdvdqtTiOZkWaieBw428wGgF3AtcB/KDvnPuA64PvALwCPqD6RYc/cfbJGYV3gk4UWiy529Wu3Oo1kQmqJwt0nzOzDwANAF3CXu//QzG4Ghtz9PuBO4K/N7EUKLYlr04pXWlQ+2sknT7YkdOETCZq14w364OCgDw0NpR2GlPrcuVVGU62F659LPp4sm2qZqftJImRmT7j7YKXntNaTJEOjnaLR7DwUkRZoCQ9JhkY7RUOrr0oKlCgkGRraGQ21zCQFShSSDPBrKFsAAAblSURBVK3xEw21zCQFqlFIcjS0s3WahyIpUItCJEvUMpMUqEUhkjVqmUnC1KIQmZL1TYFEYqIWhQhofoJIDWpRTNHdZGeLc36C/rYk49SiAN1NSuXlRaD1+Qn625I2oBYFaLZrp3vmbipvfULr8xP0tyVtQIkCNNu10z18M5X3w7LW5yfob0vagBIFaLZrp6t60fbWu4f0tyVtQIkCtA5Rp6t6MV9b+Xgj9LclbUCJAjTbtdPNdTFvZdSS/rakDWjjIhGovhlQ+aglKCQRXeylzWjjImkPce7sVm1ZjFqjlpQopEOkkijM7FbgPcBx4F+BX3b3AxXO2w4cBiaBiWrZTjpAWvMRNGpJJLUaxUPAue5+HvDPwMdrnPsudz9fSaLDpTUfQaOWRNJJFO7+oLtPFB8+Bui/TmpL685eo5ZEghj19CvAt6o858CDZvaEmW2q9UXMbJOZDZnZ0J49eyIPUlKW1p29Ri2JxDfqycz+ATijwlOfdPevF8/5JDAIvN8rBGJmq9x9t5mdRqG76r+5+3fm+t4a9dSGNPpIJFapjHpy95+r9byZXQdcCVxSKUkUv8bu4vvXzOxe4EJgzkQhbWgqGcQ16klEqkpr1NNG4GPAO9x9tMo5OWCeux8ufnwZoJXUOpl2dhNJRVo1ituAxcBDZrbFzG6HQleTmd1fPOd04Htm9jTwA+Cb7v7tdMIVEelcqbQo3P2sKsd3A1cUP94KvDHJuKTNxDlBT6SDaGa2tCdtGCQSmRCGx4pETxsGiURGiULak5beEImMEoW0Jy29IRIZJQppT1p6QyQyShTSnrT0hkhkNOpJ2pcm6IlEQi0KERGpSYlCRERqUqIQEZGalChERKQmJQoREakpto2L0mRme4CX0o6jAXlgOO0gmqC4k5PFmEFxJ62VuM90975KT7RlosgaMxuqtrNUyBR3crIYMyjupMUVt7qeRESkJiUKERGpSYkiDHekHUCTFHdyshgzKO6kxRK3ahQiIlKTWhQiIlKTEoWIiNSkRBEIM/ttM3vGzLaY2YNmtirtmOphZrea2Y+Lsd9rZkvTjmkuZvbvzeyHZnbCzIIfAmlmG83sBTN70cxuTDueepjZXWb2mpk9l3YsjTCztWb2j2b2fPFv5CNpxzQXMzvFzH5gZk8XY/5fkX8P1SjCYGanuvuh4se/Abze3T+UclhzMrPLgEfcfcLMfg/A3T+Wclg1mdlPASeAPwd+y92HUg6pKjPrAv4ZuBTYCTwOfMDdf5RqYHMws7cDR4C/cvdz046nXma2Eljp7k+a2WLgCeDqkH/eZmZAzt2PmFk38D3gI+7+WFTfQy2KQEwliaIckIkM7u4PuvtE8eFjQPB7jbr78+7+Qtpx1OlC4EV33+rux4GvAFelHNOc3P07wL6042iUu7/i7k8WPz4MPA+sTjeq2rzgSPFhd/Et0uuHEkVAzOx3zWwH8B+BLO7Z+SvAt9IOos2sBnaUPN5J4BeudmFm/cAFwD+lG8nczKzLzLYArwEPuXukMStRJMjM/sHMnqvwdhWAu3/S3dcCXwI+nG60J80Vd/GcTwITFGJPXT0xZ4RVOJaJ1maWmdki4GvAR8ta+0Fy90l3P59Ci/5CM4u0u09boSbI3X+uzlP/Bvgm8OkYw6nbXHGb2XXAlcAlHkjRq4Gfdeh2AmtLHq8BdqcUS0co9vN/DfiSu9+TdjyNcPcDZvb/gI1AZAMJ1KIIhJmdXfLwvcCP04qlEWa2EfgY8F53H007njb0OHC2mQ2Y2QLgWuC+lGNqW8XC8J3A8+7+B2nHUw8z65sabWhmPcDPEfH1Q6OeAmFmXwM2UBiN8xLwIXfflW5UczOzF4GFwN7iocdCH61lZu8D/gToAw4AW9z98nSjqs7MrgD+EOgC7nL33005pDmZ2ZeBd1JY9vpV4NPufmeqQdXBzN4GfBd4lsL/IsAn3P3+9KKqzczOA75I4e9jHnC3u98c6fdQohARkVrU9SQiIjUpUYiISE1KFCIiUpMShYiI1KREISIiNSlRiMSguArpNjNbXny8rPj4TDP7tpkdMLNvpB2nSD2UKERi4O47gM8DtxQP3QLc4e4vAbcCv5RWbCKNUqIQic/ngLeY2UeBtwGfBXD3h4HDaQYm0git9SQSE3cfN7MbgG8DlxWXCRfJHLUoROL1buAVIDOb94iUU6IQiYmZnU9hZ7q3ANcXd08TyRwlCpEYFFch/TyF/QxeplDA/t/pRiXSHCUKkXj8KvCyuz9UfPxnwOvM7B1m9l3gb4FLzGynmQW7cq0IaPVYERGZg1oUIiJSkxKFiIjUpEQhIiI1KVGIiEhNShQiIlKTEoWIiNSkRCEiIjX9f9e0nXlbqdMuAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "utils.plotBestFit(train_X,train_y, coeff)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Colic Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy =  0.6716417910447762\n"
     ]
    }
   ],
   "source": [
    "train_X, train_y = utils.loadColicData('horseColicTraining.txt')\n",
    "test_X, test_y = utils.loadColicData('horseColicTest.txt')\n",
    "coeff = logRegres.stocGradAscent1(train_X, train_y, loops = 500)\n",
    "predict_y = logRegres.classify(test_X, coeff)\n",
    "accuracy = utils.calculateAccuray(predict_y, test_y)\n",
    "print (\"accuracy = \", accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
